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About NIRN 


The mission of the National Implementation Research Network (NIRN) is to contribute to the 
best practices and science of implementation, organization change, and system reinvention to 
improve outcomes across the spectrum of human services. 


email: nirn@unc.edu 
web: http://nirn.fpg.unc.edu 
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Introduction & Purpose 


The purpose of the Drivers Best Practices Assessment (DBPA) is to assist organizations in assessing their current 
supports and resources for quality use of selected programs or practices. Specifically, organizations can use it to: 


* Identify strengths and opportunities for improvement in their current supports and resources; 
+ Select implementation best practices to strengthen staff competency and organizational practices; and 


* Provide an implementation team with a structured process to develop an action plan and data to monitor 
progress. 


Focus of the Assessment 


The Drivers Best Practices Assessment is administered for a specific practice or program, rather than for the 
organization in general. The essential functions of the program or practice should be known and clearly defined. 
It is important to choose one practice or program and answer the questions with that selected practice or program 
in mind. 


Administration Process & Key Roles 


The administration of this tool is conducted by the Facilitator who introduces the Drivers Best Practices 
Assessment. The Facilitator introduces the assessment’s purpose, provides an overview of the administration 
process and scoring, introduces concepts or big ideas measured, reads each item aloud and provides necessary 
clarification, and engages the team in the discussion and voting process. Information about key roles is provided 
in the table below: 


Facilitator 


An individual who has been trained in the administration process, has experience with the organization, and 
has a relationship with the respondent. 
The facilitator is responsible for: 


» Leading discussion and adhering to the administration process; and 


* Contextualizing items for respondents or providing examples of the organization’s work. 


The facilitator does not vote. 


Note Taker 


Key responsibility includes recording scores and ideas shared for action planning and any questions and issues 


that are raised during administration. The Note Taker does not vote. 


Participants 


Participants include implementation team members and other staff who have roles in implementation of the 
selected practice or program, are involved in different support activities or are in a leadership role for the 
organization and responsible for overseeing aspects of the implementation infrastructure. Participants vote 
on each item, discuss difference in scores, and achieve modified consensus. 


Observers are invited with permission of the implementation team to learn about the process or the activities 
in the organization. Observers do not vote. 
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Scoring 

The identified participants complete the Drivers Best Practices Assessment by discussing each item and coming 
to consensus on the final score for each item. The respondents score each item on a three-point scale (i.e., in 
place (2 points), partially in place (1 point), not in place (0 points), respectively) using a simultaneous and public 
voting process. This type of voting facilitates participation of all respondents and neutralizes any potential power 
influences. When asked to vote (e.g., “Ready, set, vote”), participants simultaneously hold up either two fingers to 
vote “Fully in Place,” one finger to vote “Partially in Place,” or a closed hand to vote “Not Yet.” Alternately, teams 
can use numbered cards to vote. If the team is unable to arrive at consensus, additional data sources documented 
in the Scoring Rubric can be used to prompt thinking and help achieve modified consensus. Modified consensus 
means that voters in the minority can live with and support the majority decision on an item. If modified 
consensus cannot be reached, the Facilitator guides the team to identify a later time for further discussion. The 
majority vote is recorded. 


Preparation for the Administration 
The following should be in place prior to administering the Drivers Best Practices Assessment. 
* Facilitator should have knowledge of the concepts measured in the assessment and experience supporting 
organizations using implementation best practices; 


* Implementation Team has agreed to administration and commitment of time (approximately one hour for 
preparation, two hours for completing the assessment, and one hour for action planning); and 


+ Materials have been assembled in preparation for administration, including: 


» Blank copies (paper or electronic) of the DBPA rubric accessible to all respondents; 
» Data sources (e.g., policies, procedures) to inform the assessment; and 


» If relevant, previously completed administrations including reports. 
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Research Basis and Outcomes from Completion 


The basis of the Drivers Best Practices Assessment is derived from implementation science research and the 
Active Implementation Framework of the Implementation Drivers (Fixsen, Naoom, Blase, Friedman, & Wallace, 
2005; Metz, Bartley, Ball, Wilson, Naoom, & Redmond, 2014). Implementation Drivers are core components 

or building blocks of the infrastructure needed to support practice, organizational, and systems change. The 
Drivers emerged from a synthesis of commonalities among successfully implemented programs and practices 
(Fixsen et al., 2005). See Appendix A for information regarding validation of the measure. 


The Implementation Drivers include: 


+ Competency- Strategies to develop, improve, and sustain practitioners’ ability to implement a program or 
practice as intended in order to achieve desired outcomes. Competency Drivers include: Fidelity, Selection, 
Training, and Coaching. 


* Organization- Strategies for analyzing, communicating, guiding, and responding to data in ways that result 
in continuous improvement of supports for staff to use the selected program or practice. Organization 
Drivers include: Decision-Support Data System, Facilitative Administration, and Systems Intervention. 
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Next Steps After Administration 


The Drivers Best Practices Assessment was created to guide organizations as they develop the infrastructure to 
support use of selected programs or practices. As such, it is recommended that teams engage in the following 
activities after they complete each administration: 


+ Review and use the (a) Total score, (b) Scale Scores, and (c) Item Scores to identify areas of strength and 
need; 


Identify priorities to address within a plan; 


Develop and create an Action Plan (Appendix B) that defines immediate and short-term actions focusing on 
improving the infrastructure activities to support use of the selected program or practice. 


If this is a repeated administration, review and update existing plan to continue support for the selected 
program or practice. 


Positive Outcomes 


* 
Fidelity 


Systems 
Intervention 


Facilitative 
Administration 


Decision Support 
Data System 


For more information on the Implementation Drivers derived by the National Implementation Research Network, 
visit http://nirn.fpg.unc.edu and the Active Implementation Hub at http://implementation.fpg.unc.edu. 
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DRIVERS BEST PRACTICES ASSESSMENT: 
Fidelity Checklist 


PROTOCOL STEPS +: Yes -:No / : Unsure or not applicable 


1. SKILLED FACILITATOR 
An individual with knowledge of implementation drivers and skill in administering the assessment is 
identified to facilitate. 


2. RESPONDENTS INVITED 
Facilitator invites participants, including Implementation Team members, who have a role in developing, 
monitoring, and improving implementation supports. 


3. PROGRAM/PRACTICE IDENTIFIED 
A well-defined program or practice is identified for the assessment. 


4. MATERIALS PREPARED IN ADVANCE 

Facilitator ensures that language in the assessment has been contextualized for the agency, copies 
(paper or electronic) are available for each participant, a note taker has been identified, and a room is 
set up with a laptop, projector, internet connection, and conference phone (video if possible) for any 
participants joining remotely. 


5. OVERVIEW 
Facilitator provides a review of the assessment, its purpose, and instructions for scoring the items. 


6. CONSENT 
Facilitator obtains informed consent from participants to collect and use their response to 
understand implementation status and inform action planning. 


7. DOCUMENTATION 
Facilitator documents date of the assessment, names and roles of participants, and the intervention 
being assessed. 


8. ADMINISTRATION & INTRODUCTION 

The Facilitator introduces the Implementation Drivers one at a time and provides an overview of the 
best practices of each Driver. Facilitator then asks the team to describe their current practices, asks 
the team which agency in the system has responsibility for the driver, and then directs the team to 
complete the items through discussion and consensus. 


9. CONSENSUS 

The team is given time to review, discuss, and come to consensus on the score for each item through a 
voting process. Facilitator answers questions, contextualizes, and provides clarification as needed for 
the respondents. The facilitator also seeks equity of voice from all participants to ensure a complete 
assessment of the context is understood. 
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NOTES 


10. RECORDING 
The team documents each scoring decision electronically or on the scoring form used to record scores. 


11. NOTE-TAKING 

For items where there is further clarity or information needed, the Facilitator notes the question in the 
“Notes” section. A note taker captures the team discussion of each Implementation Driver in the relevant 
section. 


12. DATA SUMMARY 
After the last question has been asked and answered, the Facilitator or Note Taker generates the reports 
and distributes graphs of total scores. 


13. REVIEW 

While viewing the graphs, Facilitator prompts the team in a discussion of the results to identify strengths 
and opportunities. If a repeated administration, Facilitator highlights all of the subscales that moved ina 
positive direction and celebrates progress. Facilitator initiates a discussion of updates on achievements, 
progress, and major milestones or barriers that have occurred since previous administration. 


14. PLANNING 

If there is time to review the results and action plan, Facilitator engages the team in a prioritization 
process for identifying key areas for planning and needed actions. If there is not sufficient time for 
review of results and action planning, the Facilitator ensures that a date and time are set for the 
Review and Action Planning 


15. CONCLUSION 
Facilitator thanks the team for their openness and for sharing in the discussion. 
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DRIVERS BEST PRACTICES ASSESSMENT: 
Scoring Form 


Use the Scoring Form below to capture the respondent team’s final score for each item. If the respondent team is 
unable to arrive at consensus, additional data sources for each item are documented in the Scoring Rubric. 


Facilitator(s): Today’s Date: 


Individuals Participating in the Assessment: 


Practice/Program Being Assessed Today: 


Relevant Staff for Practice/Program: 


Which staff are involved in use of the practice/program? 


Which of those staff are considered in this assessment? 


Note a rating for each item below: 2-INPLACE ~— 1-PARTIALLYINPLACE — O- NOT IN PLACE 
SELECTION 


1. There is someone accountable for the recruitment and selection of relevant staff for the 
program or practice. 


2. Job descriptions are in place for relevant staff that carry out the program or practice. 


3. Individuals accountable for selection understand the skills and abilities needed for relevant 
staff. 
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DRIVERS BEST PRACTICES ASSESSMENT: 
Scoring Form 


Note a rating for eachitem below:  2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE 


4. Selection protocols are in place to assess competencies for relevant staff that carry out the 
program or practice. 


5. Selection processes are regularly reviewed. 


TRAINING 


6. There is someone accountable for the training of relevant staff for the program or practice. 


7. Agency staff provide or secure skill-based training for relevant staff on the program or practice. 


8. Agency staff use training data for improvement. 


COACHING 


9. There is someone accountable for coaching of relevant staff for the program or practice. 


10. Coaching is provided to improve the competency of relevant staff for the program or practice. 


11. Agency staff use a coaching service delivery plan. 


12. Agency staff regularly assess coaching effectiveness. 


FIDELITY 


13. There is someone accountable for the fidelity assessments of relevant staff for the program or 
practice. 


14. The agency uses a fidelity assessment for the program or practice. 


15. Agency staff follow a protocol for fidelity assessments. 


16. Agency staff use fidelity data to improve program or practice outcomes and implementation 
supports. 


DECISION-SUPPORT DATA SYSTEM 


17. There is someone accountable for the decision-support data system. 


18. Agency staff have access to relevant data for making decisions for program improvement. 
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DRIVERS BEST PRACTICES ASSESSMENT: 
Scoring Form 


Note a rating foreachitem below:  2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE 


19. Data are useful and usable. 


20. Agency staff have a process for using data for decision-making. 


FACILITATIVE ADMINISTRATION 


21. Leadership sets aside resources to support the development of staff competency to deliver the 
program or practice. 


22. Leadership develops and/or refines internal policies or procedures that support the program or 
practice. 


23. Leadership makes changes in organization roles, functions, and structures as needed to 
accommodate the program or practice. 


24. Leadership engages in regular communication with all staff and service users regarding the 
program or practice. 


25. Leadership visibly promotes the importance of effectively implementing the program or practice. 


26. Leadership problem-solves challenges to implement the program or practice effectively. 


27. Leadership recognizes and appreciates staff contributions to implement the program or practice 
effectively. 


SYSTEMS INTERVENTION 


28. Leadership engages stakeholders and staff in developing a shared understanding of the need for 
the program or practice. 


29. Leadership creates opportunities for stakeholders and staff to learn and design solutions 
together to support the program or practice. 


30. Leadership regularly communicates with stakeholders regarding the program or practice. 
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Scoring Rubric °5LECTON 


The Selection Driver refers to use of a purposeful process for selection of staff with the required skills, 
abilities, and other program/practice-specific prerequisite characteristics. 


Tell me about your selection process(es). Record responses: 


What agency is primarily responsible for this driver? Record responses: 


1. There is someone accountable for the recruitment and selection of relevant staff for the program or practice. | | 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


A specific person is responsible A specific person is responsible There is not a specific person Job description of person 
for coordinating the quality and for coordinating the quality and responsible for coordinating accountable for recruitment 
timeliness of recruitment and timeliness of recruitment and the quality and timeliness of and selection 

selection processes for relevant staff | selection processes for relevant staff | recruitment and selection processes 

supporting the program or practice. | supporting the program or practice. | for relevant staff supporting the 

This person is able to execute the program or practice. 

responsibilities related to his/her 

role in the selection process. 


2. Job descriptions are in place for relevant staff that carry out the program or practice. | | 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Job descriptions are: Job descriptions are clear about Job descriptions are not clear about | Job descriptions 
+ clear about expectations for the expectations for the position. expectations for the position or 
position; aligned with the competencies. 
* aligned with the competencies 


required for the program to be used 
competently. 
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3. Individuals accountable for selection understand the skills and abilities needed for relevant staff. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Individuals accountable for 
selection: 
+ know the knowledge, skills, and 
abilities related to the staff position; 


* accurately assess applicant 
knowledge, skills, and abilities. 


4. Selection protocols are in place to assess competencies for relevant staff that carry out the program or practice. 


Individuals accountable for 
selection: 
+ know the knowledge, skills, and 
abilities related to the staff position. 


Individuals accountable for selection | Job descriptions 
have little or no knowledge of the 
knowledge, skills, and abilities 
related to the staff position. 


Selection protocol 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Selection protocol includes all of the | Selection protocol includes all of the | Generic selection protocol (e.g., 


following: 
* an assessment of core skills needed 
for position; 


specific procedures (e.g., scenario, 
role play) for assessing individual’s 
ability to perform key skills; 


specific procedures for assessing 
ability to receive and use feedback 
provided during the interview; 


a documented process for review of 


adherence to the interview protocol; 


record of the ratings of individuals’ 
responses. 


following: 
* an assessment of core skills needed 
for position; 
* adocumented process for review of 
adherence to the interview protocol; 


* record of the ratings of individuals’ 
responses. 


5. Selection processes are regularly reviewed. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Selection processes are annually 
reviewed and revised as needed to 
improve the selection process. The 
annual review examines at least 
three of the following: 


interview results (e.g., protocol 
adherence, applicant responses); 


training data; 
turnover data; 
fidelity data; 

exit interview results. 


Selection processes are annually 
reviewed and revised as needed to 
improve the selection process. The 
annual review examines at least one 
of the following: 

interview results (e.g., protocol 

adherence, applicant responses); 

training data; 

turnover data; 

fidelity data; 

exit interview results. 


Selection protocol (including 
procedures used during the 
selection process) 


similar protocol for any position) 
exists. 


Data showing the results of 
core skills assessments 


Selection processes are not 
reviewed and revised. 


Selection process 
documentation 


Data on selection outcomes 


Scoring Rubric 


TRAINING 


The Training Driver refers to use of purposeful, skill-based, and adult-learning informed processes designed 
to support relevant staff in acquiring the skills and information needed to support the program/practice. 
Training of relevant staff at the agency provides knowledge related to the theory and underlying values of 
the program/practice, opportunities to practice new skills to meet fidelity criteria, and feedback in a safe and 
supportive training environment. 


Tell me about your training process(es). Record responses: 


What agency is primarily responsible for this driver? Record responses: 


6. There is someone accountable for the training of relevant staff for the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


A specific person is responsible for 
coordinating quality and timeliness 
of training for relevant staff 


supporting the program or practice. 


This person is able to execute the 
responsibilities related to his/her 
role in training. 


A specific person is responsible for | There is not a specific person Job description of person 
coordinating quality and timeliness | responsible for coordinating accountable for training 
of training for relevant staff quality and timeliness of training 
supporting the program or practice. | for relevant staff supporting the 

program or practice. 
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7. Agency staff provide or secure skill-based training for relevant staff on the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Training is: Training is: Training is not: Professional learning 
« required and provided before staff * required and provided before staff * required and/or is not provided schedule 

begin to use the program or practice; begin to use the program or practice; before staff begin to use the new _ 
provided by trainers who have deep | « provided by trainers who have deep program or practice; Training outlines or agendas 
content knowledge and who are content knowledge and effective * provided by trainers who have deep _ ; 
effective trainers; presentation delivery skills. content knowledge and effective Training evaluations 
skill-based and includes presentation delivery skills. 
opportunities for practice and Presenter qualifications 
feedback in a safe environment; _ 
comprehensive, including practice- Agendas for training 
specific and complementary skills presenters 
(e.g., equity, diversity, and inclusion). 


8. Agency staff use training data for improvement. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Training assessment data are: Training assessment data are: Training assessment data are not Training outcome data 
+ collected and used to improve future | + collected and used to improve future | collected. ; 
training activities; and training activities. Evidence that data are used 
* provided to supervisors and for improvements 
coaches in a timely manner to 
improve staff competency and other 
implementation drivers. 


Scoring Rubric COACHING 


The Coaching Driver refers to the purposeful process of supporting staff to generalize newly learned skills for 
the program/practice, to be used competently by the practitioner in real-world settings and interactions. 


Tell me about your coaching process(es). Record responses: 


Which agency is primarily responsible for this driver? Record responses: 


9. There is someone accountable for coaching of relevant staff for the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


A specific person is responsible A specific person is responsible There is not a specific person Job description of person 
for coordinating the quality and for coordinating the quality and responsible for coordinating the accountable for coaching 
timeliness of coaching relevant staff | timeliness of coaching relevant staff | quality and timeliness of coaching 

supporting the program or practice. | supporting the program or practice. | relevant staff supporting the 

This person is able to execute the program or practice. 

responsibilities related to his/her 

role in the coaching process. 


10. Coaching is provided to improve the competency of relevant staff for the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Coaching is provided at least Coaching is provided at least Relevant staff do not receive Coaching schedules 
monthly to relevant staff. Coaches’ | monthly to relevant staff. Coaches’ | coaching at least monthly. 
feedback to staff is based on direct | feedback to staff is based on one of Samples of coaching feedback 
observation and at least one other _| the following: 
data source such as: * group or individual reflections; 
* group or individual reflections; * product or document review; 
* product or document review; - fidelity data; 


* fidelity data; - interviews with key stakeholders. 
+ interviews with key stakeholders. 
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11. Agency staff use a coaching service delivery plan. 


2- IN PLACE 


Awritten plan outlines coaching 
provided to relevant staff, including 
three of the following: 


skill sets for being a coach; 
frequency of coaching; 

coaching methods; 

feedback methods and timeframe; 


communication protocols for coach 
and supervisor. 


Adherence to the plan is reviewed 
regularly. 


12. Agency staff regularly assess coaching effectiveness. 


2- IN PLACE 


Agency staff assess coaching 
effectiveness quarterly through the 
use of two or more data sources: 

* practitioner fidelity; 

* coach fidelity; 


* staff satisfaction with coaching 
surveys. 


Coaching effectiveness data are 
used to improve coaching and other 
implementation drivers. 


1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Awritten plan outlines the coaching | A written coaching service delivery | Sample of coaching service 
supports provided to relevant plan does not exist. delivery plans 
staff, including at least one of the 
following: Content and concept lists 
« skill sets for being a coach; used by coaches 
frequency of coaching; 
coaching methods; 
feedback methods and timeframe; 


communication protocols for coach 
and supervisor. 


1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Agency staff assess coaching at least | Coaching effectiveness is not Coaching fidelity: 

annually through the use of at least | assessed. * observations of coaches 

one data source: conducting coaching activities 
* practitioner fidelity; * coaching logs 
* coach fidelity; 
* staff satisfaction with coaching 

surveys. Satisfaction surveys from 
those being coached 


* coaching notes 


Evidence the data are used to 
inform improvements in 
coaching methods 


Scoring Rubric [DEUTY 


The Fidelity Driver refers to the purposeful process of using fidelity assessments to evaluate the extent to 
which a program/practice is implemented as intended. 


Tell me about your fidelity process(es), including how often fidelity data are reviewed. Record responses: 


Which agency is primarily responsible for this driver? Record responses: 


13. There is someone accountable for fidelity assessments of relevant staff for the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


A specific person is responsible for | Aspecific person is responsible for 
coordinating fidelity assessments coordinating fidelity assessments 
of relevant staff for the program of relevant staff for the program or 
or practice. This person is able to practice. 

execute the responsibilities related 

to his/her role. 


14. The agency uses a fidelity assessment for the program or practice. 


There is not a specific person Job description of person 
responsible for coordinating fidelity | accountable for fidelity 
assessments of relevant staff for the | assessments 

program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


The agency consistently uses a The agency inconsistently uses a 
fidelity assessment for the program | fidelity assessment for the program 
or practice. or practice. 


The agency does not use a fidelity Fidelity assessment may 

assessment. include multiple measures to 
address context, content, and 
competency 


Technical manual 


Research documents 
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15. Agency staff follow a protocol for fidelity assessments. | | 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Agency staff follow a written Agency staff follow a written Agency staff do not follow awritten | Fidelity assessment 
protocol that includes all of the protocol that includes some but not | protocol for fidelity assessments. protocol 
following: all of the following: 
orientation process for relevant staff; orientation process for relevant staff; Documentation of fidelity 
process for how fidelity data are process for how fidelity data are assessments 
used; used; 


communication protocol for sharing communication protocol for sharing 
fidelity data. fidelity data. 


16. Agency staff use fidelity data to improve program or practice outcomes and implementation supports. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Agency staff review fidelity Agency staff review fidelity Agency staff do not review or use Documentation of action 
assessment data regularly and assessment regularly but data are fidelity assessment data. plans for improvement of 
use assessment data to improve are used inconsistently to improve selection, training, or 
implementation drivers. implementation drivers. coaching processes 


Documentation of feedback to 
coaches and/or trainers 


Documentation of feedback 
provided to practitioners 


Scoring Rubric DECISION-SUPPORT DATA SYSTEM 


The Decision-Support Data System refers to the development and use of data systems to support decision 
making and improvement activities, including the collection and use of programmatic data, fidelity data, and 
outcome data. 


Tell me about your decision-support data system process(es). Record responses: 


Which agency is primarily responsible for this driver? Record responses: 


17. There is someone accountable for the decision-support data system. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


A specific person is responsible for |Aspecific personis responsible for | Thereis no person responsible for Job description of person 
coordinating a data system that is coordinating a data system used coordinating a data system used accountable for decision- 
used to support decision-making to support decision-making for to support decision-making for support data system 


for the program or practice and the program or practice and its the program or practice and its 
its implementation. This person is implementation. implementation. 

able to execute the responsibilities 

related to his/her role in overseeing 

the decision-support data system. 


18. Agency staff have access to relevant data for making decisions for program improvement. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Relevant staff have access to and Relevant staff have access to and Relevant staff do not have access Sample data reports 
can analyze all of the following data | can analyze some but not all of to any of the following data for 
for program improvement: the following data for program program improvement: 

fidelity data; improvement: fidelity data; 

outcome data; fidelity data; outcome data; 

programmatic data, including outcome data; programmatic data, including 

feedback from practitioners and programmatic data, including feedback from practitioners and 

program beneficiaries; feedback from practitioners and program beneficiaries; 

financial data. program beneficiaries; financial data. 

financial data. 
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19. Data are useful and usable. 


2- IN PLACE 


Data collected meet all of the 
following criteria to be useful and 
usable: 
collected in a standardized way by 
trained staff; 
provide relevant information that 
can support improvement processes; 
available when relevant staff are 
making decisions; 
an important component of practice 
routines. 


1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Data collected meet some but not 
all of the following criteria to be 
useful and usable: 
* collected in a standardized way by 
trained staff; 


* provide relevant information that 


can support improvement processes; 


¢ available when relevant staff are 
making decisions; 

* an important component of practice 
routines. 


20. Agency staff have a process for using data for decision-making. 


2- IN PLACE 


Agency staff have a process for 
using data for decision-making that 
includes all of the following: 


data are disaggregated, analyzed, 
and summarized at least quarterly; 


data summaries are communicated 
clearly in written reports to relevant 
staff; 


action plans are developed and 
monitored regularly to improve 
implementation supports and 
outcomes; 


data summaries and action plans are 
shared with key stakeholders. 


Agency staff have a process for 
using data for decision-making that 
includes some but not all of the 
following: 

+ data are disaggregated, analyzed, 
and summarized at least quarterly; 
data summaries are communicated 
clearly in written reports to relevant 
staff; 
action plans are developed and 
monitored regularly to improve 
implementation supports and 
outcomes; 
data summaries and action plans are 
shared with key stakeholders. 


Data collected do not meet any of 
the following criteria to be useful 
and usable: 
* collected in a standardized way by 
trained staff; 


* provide relevant information that 


can support improvement processes; 


¢ available when relevant staff are 
making descisions; 


* animportant component of practice 


routines. 


Agency staff do not have a process 
for using data for decision-making. 


Sample data team meeting 
notes 


1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Documentation of processes 
used by agency to review data 
and make decisions 

Sample data reports 


Sample action plans 


Scoring Rubric FACILITATIVE ADMINISTRATION 


Facilitative Administration refers to an agency’s leaders, managers, and implementation teams developing 
and using strategies that facilitate and support use of the program/practice, and that make the work of 
practitioners easier. For the purpose of this assessment, leadership is inclusive of your executive leaders, 
managers, and team members who are responsible for the program or practice. 


Tell me about your agency’s/site’s organizational structure (e.g., leadership, management, teams). For the purpose of this assessment, 
leadership is inclusive of your executive leaders, management, and team members who are responsible for the program or practice. 
Record responses: 


21. Leadership sets aside resources to support the development of staff competency to deliver the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership sets aside resources Leadership sets aside some but Leadership does not set aside Budget 
to support staff competency not all resources to support staff resources at all or does so in general 
development: competency development: (i.e., not for the specific program/ 


- selection » selection practice). 
* training * training 

* ongoing coaching, and * ongoing coaching, and 

* monitoring fidelity * monitoring fidelity 


22. Leadership develops and/or refines internal policies or procedures that support the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership consistently develops Leadership develops and/or Leadership does not develop and/ Budget 
and/or refines policies and refines policies and procedures or refine policies and procedures to 
procedures to make it possible to do | inconsistently. make it possible to do the work of Training 
the work of the program or practice. the program or practice. 


Resources 


23. Leadership makes changes in organization roles, functions, and structures as needed to accommodate the 
program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership consistently makes Leadership inconsistently makes Leadership does not make changes __| Organizational chart 
changes to organization roles, changes to organization roles, to organization roles, functions, and 
functions, and structures. functions, and structures. structures. Position descriptions 
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24. Leadership engages in regular communication with all staff and service users regarding the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership communicates with all Leadership communicates to all staff | Leadership does not communicate | Communication plan 
staff and service users and receives | and service users. regularly with staff and service 

and responds to feedback from all users. Example communications 
staff and service users. 


25. Leadership visibly promotes the importance of effectively implementing the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership speaks about and can Leadership speaks about the Leadership struggles to speak Communication plan 
answer questions regarding what it | importance of implementing the about and answer questions 
takes to effectively implement the program or practice but struggles to | regarding what it takes to effectively | Example communications 
program or practice. answer questions about what it will | implement the program or practice. 

take to do so effectively. 


26. Leadership problem solves challenges to implement the program or practice effectively. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership consistently problem- Leadership inconsistently problem- | Leadership does not problem-solve | Meeting minutes 
solves challenges using data to solves challenges using data to challenges using data to effectively 

effectively implement the program | effectively implement the program | implement the program or practice. | Observations 

or practice. or practice. 


27. Leadership recognizes and appreciates staff contributions to implement the program or practice effectively. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership consistently recognizes | Leadership inconsistently recognizes | Leadership does not recognize and | Meeting minutes 
and appreciates staff contributions | and appreciates staff contributions | appreciate staff contributions to 

to effectively implement the to effectively implement the effectively implement the program | Observations 
program or practice. program or practice. or practice. 


Scoring Rubric SYS'EMS INTERVENTION 


Systems Intervention refers to how agency leaders, managers, and implementation teams work with diverse 
and representative external partners. These partners include funders, organization’s board or governing 
entity, beneficiaries of the practice or program, and community partners. Leadership works with these 
partners to ensure availability of resources required to align and deliver the practice. 


Tell me about your systems intervention process(es). For the purpose of the assessment, stakeholders are 
external groups who are necessary for the successful use of the program or practice. Record responses: 


28. Leadership engages stakeholders and staff in developing a shared understanding of the need for the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership works together with all | Leadership works together with at —_| Leadership does not work with Meeting minutes 
of following stakeholder groups and | least one of following stakeholder stakeholder groups and agency staff 
agency staff to develop a shared groups and agency staff to develop | to develop a shared understanding | Communications 
understanding of the need for a a shared understanding of the need | of the need for a program or 
program or practice. for a program or practice. practice. 

« Funders and/or Board « Funders and/or Board 

* Beneficiaries of the practice or * Beneficiaries of the practice or 

program (e.g., clients) program (e.g., clients) 
* Community Partners * Community Partners 


29. Leadership creates opportunities for stakeholders and staff to learn and design solutions together to support the 
program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership creates opportunities Leadership creates opportunities Leadership does not create Meeting minutes 
consistently to learn and design inconsistently to learn and design opportunities to learn and design 

solutions together to support the solutions together to support the solutions together to support the Agendas 
program or practice. program or practice. program or practice. 


Communications 
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30. Leadership regularly communicates with stakeholders regarding the program or practice. 


2- IN PLACE 1- PARTIALLY IN PLACE O- NOT IN PLACE DATA SOURCE 


Leadership Leadership provides information to | Leadership does not engage in Communications 
* provides information to stakeholders | Stakeholders regarding the program | communication with stakeholders 
regarding the program or practice; or practice. regarding the program or practice. Meeting minutes 


* receives information from ; 
stakeholders regarding the program Observations 
or practice; 

* requests and responds to feedback 
from all stakeholders regarding the 
program or practice. 
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Appendix A 
Validation 


Ogden et al. (2012) at the Atferdssenteret - Norsk senter for studier av problematferd og innovativ praksis - 
Universitet i Oslo (The Norwegian Center for Child Behavioral Development, University of Oslo) validated a 
previous version of the Drivers Best Practices items. Ogden et al. collected data to establish the reliability and 
validity of the Implementation Driver items. The researchers interviewed 218 practitioners, supervisors, and 
managers associated with two well-established evidence-based programs in Norway. The Cronbach alphas 
obtained in their study were: selection, 0.89; training, 0.91; coaching, 0.79; fidelity, 0.89; decision-support data 
systems, 0.84; facilitative administration, 0.82; systems intervention, 0.82; and leadership, 0.88. 


Metz et al. (2014) assessed Implementation Drivers in a county social service system before, during, and after 
implementation capacity was developed. Low scores on the Drivers assessment at baseline were associated 
with low levels of fidelity use of the innovation. As implementation capacity was developed, the scores on the 
Drivers assessment increased (nearly doubled). Higher scores on the Drivers assessment were related to much 
higher fidelity use of the innovation. 
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Appendix B 


Drivers Best Practices Assessment Action Plan Template 


Contributors to Action Plan: Date of Action Plan: 


Focus of DBPA: 


Based on your review of the DBPA results, identify at least 2-3 priorities to address within an action plan. 
Create an Action Plan using the template below that defines immediate and short-term actions focusing on 
improving the infrastructure activities to support use of the selected program or practice. 


BY WHO BY WHEN 
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